In the field of computing hardware and software technology, users access and engage with a variety of online services with increasing regularity. Familiar examples of online services include, but are not limited to, email, social networking, media consumption, word processing, gaming, and file storage services. A typical online service runs in a data center or within the context of some other compute resource. A local client application exchanges information with the online service over a network or networks to facilitate what a user may experience as the features and functionality of the service.
At least some of the information exchanged between an online service and a client application includes the objects, data, images, components, and other interface elements with which a user interface to the online service may be rendered. The client application, usually with the cooperation of other applications or modules executing locally, processes the interface elements in order to present a coherent user interface with which a user may interact to enjoy the features and functions of the service.
The look and feel of a user interface, and even which features and functions are experienced in the user interface, is largely governed by how a service is implemented during the build and deployment stages of service provisioning. Many online services, especially those that operate at scale, utilize many developers to create the underlying program code for an online service. The developers generate the code which is then submitted to a build environment to be compiled and integrated into a form that can be installed in a deployment environment. The configuration of a user interface is thus set at the time its associated code is submitted to a build environment.
However, experimenting with different configurations for a user interface can be beneficial for developers. In order to accomplish such experimentation, various configurations of the user interface are coded during development and submitted to a build environment. In addition, parameters for an associated experiment are also coded and submitted to the build environment. The resulting version of the service that is ultimately deployed in the deployment environment thus includes the differing configurations of the user interface and the associated experimental parameters. At run-time, an experiment can be conducted whereby the various user interface configurations are served to end users per the experimental parameters established at build-time.
In an example, two different user interface configurations may be coded by developers and submitted to a build environment, along with experiment parameters indicating that the first configuration of the two is to be served to a quarter of all users while the other configuration is to be served to the remaining three-quarters of all users. The user interface configurations and experiment parameters will then be processed in the build environment and loaded into the deployment environment. Subsequent users will experience the two configurations in accordance with the experiment parameters.
In order to change the experiment, a new build of the service must be produced in the build environment and then provisioned in the deployment environment. In order to change even just the ratio of users served one or the other of the two configurations in the example above, a new instance of the service with the new experiment parameters would have to be submitted to the build environment and then deployed. In some cases there may be configurations that were built and deployed but that are not in use generally, nor are in use with respect to an experiment.